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1.0) Introduction and Overview 

The purpose of the research effort presented here is to prescribe a generic 
reusable shell that any project office can install and customize for the purposes of 
advising, guiding, and supporting project managers in that office. The prescribed 
shell is intended to provide both: (1) a component that generates prescriptive 
guidance for project planning and monitoring activities, and (2) an analogy 
(intuition) component that generates descriptive insights of previous experience 
of successful project mangers. The latter component is especially significant in 
that it has the potential to: (a) retrieve insights, not just data, and (b) provide a 
vehicle for expert PMs to easily transcribe their current experiences in the course 
of each new project they manage ( i.e. to act as the Corporate Memory). 

For the past several years the principal author has conducted psychological, 
behavioral, and cognitive studies of expert project managers' thought processes 
for the purposes of deriving a model suitable for translation into an expert 
system. The model is based on the process of diagnosis and analogical reasoning 
as described above and in sections of this paper. This model is based on the study 
of 21 employees of NASA, numerous employees of the U.S. military, historical 
case studies from the Space station and Space Telescope Programs and papers of 
16 famous inventors (e.g., Ben Franklin's diaries, to mention one) as documented 
in earlier reports. It is expected that the successful implementation of the model 
and the integration of the analytical and analogical components will result in 
many new innovations including special-purpose expert system generators, which 
would represent a new phase in the maturation of Expert Systems technology for 
project management applications. 
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1-1) Technical Objectives 

The focus of this paper will be to report on the preparation conduct and 
results of an experiment to prove/disprove the premise that an expert project 

msnnpf ?n ent s / stem can be c° nf| gured that will improve/expand the ability of a 
manger to perform project planning and monitoring. This experiment has^een 
designed with the intention of accomplishing the following three objectives: 

(1 * Simplified Prototype containing a Project Management 

Lcr • + ar > analogical reasoning inferencing mechanism and the 

associated knowledgebases. 

I 2 * Exploration o f Elev en Key Research Questions relating to the nature of an 
expert project management system (EPMS) environment. 

EP ^ alUatl ° n ° f the Prototvpe and Rec ommendation of Design Guidelines for 
Version 1, 

The evaluation of the prototype will consist of a system 

°*' mance eva luation based on snapshots and backtracing of actual EPMS runs 

f l ^°coc C c°!T men T S k SU ^ 9eSt [ 0nS 12 ex P Grts Presented with three exemplary EPMS 
user sessions. The insights obtained from these evaluations will be used to 
formulate design guidelines for a working Version 1 system, which is expected to 
beyond the current limits of expert system shells, and Ixhfbit the 
characteristics of an expert system kernel or generator. 

1- 2) Report Organization 

- Tbls report will present in succession, the framework and results of the activities 
aimed at the accomplishment of the three technical objectives The next section 

EPMS nnnV,mr,r nO wf dSe s el ' C ' tatl0r1 PTOCeSS and the r «P | t'"9 framework for the 
, Section 3 contains a top-level description of the prototype and 

the evaluation of the prototype will be presented in section 4. The last section 
presents the conclusions reached and outlines of planned future developments. 

2- 0) PM Knowledge Elicitation 

knowledge" describes the concept ° f EPMS that evolved over dozens of 

collection sessions. In each session, feedback from domain experts was solicited bv 
giving demonstrations and/or functional descriptions of EPMS: i.e. its goals its 
conceptual design, and the types of sessions a user would encounter. 

A common observation among the experts was the need to implement EPMS 

rJktinr? ny 9 ' Ven ! wo ' l f V u e hlGrarch y' m order to be compatible and supportive of 
existing organizational boundaries and lines of communication (see Fiqure 1) 
Figure 1 also reflects the perception that the manager probably will no? be the 
prmcipal user of EPMS and that is more realistic to expect an Executive Assistant to 
assume the user role, and to expect the manager and submanager^to ule EPMS 
either indirectly through the Assistant or occasionally themselves. 

A significant observation made during the knowledge elicitation sessions was 
the presence of a wide diversity of needs for stand-alone expert sySeTba^ed 
project management support tools. One of the managers interviewed presented 1 

st w^T e °^ the P° ssib,e , areas f o. r ES support (see taBle 1) and indicated that this 
list was by no means exhaustive. Furthermore, there was found to be an 
Qverwhelming preponderance of existing subsystems, data bases, MIS DSS etc 
which would require direct interfacing to an integrated EPMS Kernel,' or would 




- 2 - 


c 


ORIGINAL PACE B 

OF POOR QUALITY 


Table 1 : Summary of Possible Stand-alone PM Expert Systems 


Estimating 


Cost and Time Control Site Planning and Management 


Project Selection 

Conceptual Estimating 
Parameter Estimating 
Life Cycle Costing 
Value Engineering 
Integration with CAD 
Historical Cost Data Management 
Collection 
Maintenance 
Tailoring 

Check List 

Contract Document 
Site Inspection 

Start Procedures Manuals 
Quantity Survey 
Pricing 

Securing Material and Subcontractor 
Prices 

Proiect Conceptual Planning 
Equipment Analysts 
Crew Analysis 
Cash Flow Analysis 

Bid Preparation 
Sub Bid Analysis 
Sub Bid Scratch Estimates 
Material Price Analysis 
Markup Analysis 
Bid Adjustments 


Planning 

Activity Breakdown 
Logic Definition 
Duration Analysis 
Contingency analysis 

Input Data 

Collections 

Checking 

Activity and Project Status 
Analysis 

Overrun Projections 
Problem Flagging 
Problem O agnosis 
Remedy Recommendations 

Changed Condition 
identification 
Impact Evaluation 
Recommendat ons for Execution 
Notifications of all Parties Involved 

Productivity Analysis 

Progress Payment Application Preparation 
invoice Checking 
Material Tracking 

Penalty/ligu'dted Damages Evaluation 


Site Layout 
Materials Handling 
Temporary facility Requirements 
Site Management Staffing 
Access and Traffice Control 
Security Systems 
Safety Systems 
Proiect Closeout 

Retention Reduction and F«nai Payment 

Resource Reduction 

Closeout Documentation Check '*st 

Subcontract Closeout 

l-en Release 

Project Debriefing 

Historic Cost Acquisition 
Productivity Analysts 
Learning from the Project 


Change Order Estimating 
Impact Analysts 
Change Condition Pricing 


General Conditions Development 
item Selection 
Pricing 


insurance Analyse 


Figure 1 EPMS Two-Tier Hierarchy 
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require data/knowledge transfer in the case of a stand-alone EPMS. In either 
+k U + atl i° n ' com .P a ‘ t| b[l , ty with existing resources emerged as an important criterion 
that places unique flexibility demands on the expert system "shell" . 

In response to the need for this adaptability a concept for an EPMS generator 
having a four-ringed architecture was adopted (see Figure 2). 

2-1) Ring Four: Site Specific Elements 

. The outermost ring is representative of a gateway to the manager's external 
information environment. Most of the managers interviewed indicated a strona 
dependence on the availability of reference information, historical data and other 
large data base management and information retrieval requirements. Access to the 
external environment is accomplished in many different ways including person-to- 
person communications, on-line retrieval via a computer terminal, customized 
research conducted by a services firm, or physically locating the information in a 
library or other repository. Most of the groups indicated that for an EPMS 
generator to be effective, this vast array of information resources had to be taken 
into account, either bv direct interface (in the case of computer data bases) or at a 
minimum, by identifying the source, point of contact, and location where 
supplementary information can be obtained. In essence, the outer ring represents 
the various hooks" of the EPMS generator to the outside world, including 
intelligent information retrieval, organizational knowledge, generation of 
copies/sessions for use on proliferated stand-alone machines, and numerous other 
extension utilities. 


2.2) Ring Three: The PM Kernel 

The next ring represents the next "layer" of project management activity that 
emerged as a result of the experts' discussions. PM activity was found to have two 
main modes: 1) planning, where specifications, budgets, milestones, etc. for a new 
project are formulated, and 2) monitoring, where the execution of the plans 
developed in (1) are carried out. Most participants indicated that, after they had 
researched and obtained the necessary (or at least the most available) reference and 
background material regarding a problem or decision, the next step involved a 
series of processes where the information was sorted, ordered, analyzed and 
presented. Performance of this type of activity was the basis for the desiqn of the 
planning mode of the EPMS generator. This generator consists of a project 
management subsystem that contains heuristics and analytical techniques used by 
project managers in analyzing information, assessing problem situations and 
generating proposed responses. For the monitoring mode, it was necessary to make 
available a subsystem of customizing utilities, whereby a project manager could 
specify and create an automated layer of information filtering" including 
parameter and alarm thresholds, milestones, quick-look summaries etc Finally a 
user interface subsystem that makes use of human factors and computer visual 
engineering (CVE) principles was identified as a requirement for both modes. The 
interface design feature most requested by the experts was the ability to choose 
from a list of presentational formats, depictions, or other customized user- 
generated displays. 


2.3) Rings Two and One: Analogical Reasoning Applied 

The analytical filtering and processing specified for Ring Three is intended to 
result in the generation and display of key indicators, barometers and other 
parameters that are important to project management decision making. Most 
managers agreed that it was at this point in the thought process that analogical 
reasoning was most frequently applied. This was evident in most manager's 
comments, which stated that comparisons with past similar experiences were keyed 
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MnS th 3 rT^ ter ^ reS *k ing aft u r exte " sive analysis of the data had been performed 
(ring three) rather than on the raw data elements themselves (ring four). 

The process of analogical reasoning is best understood by referrina to the 

nfohl^m 3 a 3 y teSt q V estlon w hich is invariably written in the form: "some known 
p oblem, A, is to some known solution, B, as a new problem, C, is to which of several 

Th'c SI, c w 0F) ^ 0 2 S(X i^' or Z ) ? more tersely, this can be written as A:B:C(X YZ)? 
This is depicted in Figure 3, where A and B are the Base problem-solution pair (or 

Satement°X Y^or Z°are th?'^ ana '° + 9 eX ' StS) , and wherG C is the tarqet Problem 
circle 6t X,Y * ° Z arethe unk nown target solution shown in Figure 3 as an empty 

A traditional stand-alone expert system operates on target problem-solution 

pairs generally via a series of productions of the form: 

THEN:? 

' S ' tFie trad l tional exp f rt system isolates the conditions of the target 
problem space and attempts to directly infer the solution to be one of X,Y,or Z. 9 

a«JlL?!5i er t0 make Gffective use of analogy, particularly in an automated 
a " a 5 fh re3S ^T 9 Sup , port environment, studies of the knowledge elicitation 
sessions showed the need to extend a traditional expert system in two DrinciDal 
ways: (1) facilitate identification of the target problem. C,by lookinq fj? s?mMar 
problem statements in the set of bases. A, and (2) to help flush out the Walt 

^w Ce h by a SU9 ? eStm9 P !. St solutio " s from 8 th ^t in part or in whole appear 
| P M anc, hy assisting in adapting those solutions to the current problem 

ad uswen?" h^f ° Th S t0 re l!° + ° he n V,ly ° n l he past " '■*- the "anchoring and 
adjustment bias). This capability is illustrated as the "extended expert system 

!n CU h- h h ?h Wn + m Fl ? ure u 3 ; Fur thermore, this expanded focus must support situations 
k Pr ° b em< C< ' S '? lt,ally 'H-defined, rather than known a priori, as 

uJ ^mp n ^h| h o m0 w e + C0nVentl ° na anal °9V programs. The same uncertainty must 
ttficTfr 9 b determining analogous problem-solution pairs (A and B)s For 
this reason a major goal of this development effort was to gain the ability to scan a 
large set of possibilities and to generate and test ideas, with the best idea° beinq 

lM a mma n ^n f h°;uds e ti?s n9 ' manipu,atl ° n ' transformation, and other disanalog? 
3.0) EPMS Prototype 

Rgure 4 provides an overview of the portions of EPMS that were experimented 
w ' th ‘P the prototype. There are three major parts to Figure 4 - the lonqer term 
aids the core of EPMS, and the customizing utilities and user support functions Vhe 
?nn,?nh P nf' / It S J n ex Pf'™ent upon the latter two parts which involved building just 
nprt U iHprS f A 3C !k P 1 rt to glean insights useful for next step development. Th efirst 
p *2 vf ntd,es the l° n 9 term aids tnat are foreseen in order to integrate EPMS into 

tnnpThi^th SUP c^/ic StrUCture 'u Vhlch would complete the kernel concept by bridging 
together the EPMS core with external data bases, tools, algorithms sources of 
knowledge, workstations, personal computers, and other hardware. 

The customizing utilities identified on the right-hand side of Figure 4 are geared 
toward m a kmg the EPMS kernel attractive to a wide variety ofpoteniaPSSs 
ence the utilities support the tailoring of each and every knowledqe base analoa 

Ate.otih? SU ? SyStem m ° d , ule ,0 s P ecifk a PP llca ^ 10 ^^ the in d iv^du user* 
Although the customizing utilities were not developed for the prototype their 

design and scope was a major focus of the experiment. High level designs for many 


- 6 - 



©f-: POOR QUALITY 


( 


Figure S Project Management Subsystem Overview 
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Figure 6 An Inielligent Subsystem Milestone Aspect Object of the PMKB 


[ Subsystem-Mi 1 est one -Aspect Name : 

Def imtion: 

Comment : 

Objects Depended On: 

Objects Af fectabl e : 

Techniques To Use For Inferring The Plan: 
Plan Value: 

Comment on Value: 

Technique Used: 

Date By Which Plan Value Is Due: 

Techniques For Determining Actual Value: 
Techniques For Projecting Actuals: 

Actual Value (Current Date): 

Actual Value (Projected To Oue Date): 
Caution Threshold Level: 

Object Status (one of Completed, On Track, 
Caution, Emergency): 

Actual Value Comment : ] 


( 


( 



Figure 4 EPMS Elements Exp merited With in the Prototype 






















we^efi cited were createb anb user evaluations of and reactions to these designs 

The remainder of this section will be devoted to providing descriptions of the 
four main components of the EPMS core, along with a discussion of the three 
exemplary user sessions that were run during the course of the experiment. 

3.1) PM Subsystem 

An overview of the PM subsystem is shown in Figure 5, which further illustrates 

D 6 CO /DfJPJm f °P® rat,on L ln the planning mode, a self-contained PM 
Knowledge Base (PM KB) works with a stand-alone backward-chaining inference 
engme to assist the user in planning all aspects of a new project. The inference 
engine knows how to draw on the ARIEL subsystem for assistance if its stand- 
alone techniques are unable to estimate a value required by the plan or if a search 
across a wider range of candidate analogs is required. That is, it chains through 
each subsystem, milestone, and aspect of the new plan for a new project. Tne 
aspects elicited include planned levels of manpower, dollars, etc. per time period 
and work package. y 

In the monitoring and control mode, a forward chaining inferencing technique is 
used in conjunction with active value "demons" to constantly monitor and test the 
deviations of actual values from planned values for the various subsystem- 
milestone-aspect objects. When cautionary (or emergency ) alerts are detected all 
re /5u J ect * Wltb ' n * be knowledge base are tagged with an alarm message 
which allows the user to determine the source and the nature of any deviation that 
may affect overall project performance. This cross-referencing feature was cited as 
a major requirement currently lacking in most project monitoring systems. This 
mode was also equipped with a clock and calendar, in response to concerns 
expressed regarding the lack of the ability of current expert systems to adequately 

a ^°“ nt ' or rn?^ ects tbe P assa 9 e of time on any given situation. As a result all 
activities in EPMS are time-tagged in a Julian date format, as a means of keeping a 
record of the time of occurrence and duration of important events. The operation 
ot each subsystem component including the control panel will be illustrated further 
in the description of the user sessions. 

3.2) Project Management Knowledge Base (PMKB) 

The EPMS PM Subsystem seeks to establish a new project plan and to monitor its 
progress. This is done cooperatively and interactively with a human participant and 

2^TL eted D proj f A c t^ la ^ ult,mate| y becomes one more analog in the Analog 
Knowledge Base (AKB). The PMKB is thus a set of rules and objects designed to 
capture and hold the subsystem-milestones-aspect knowledge for the "tarqet" 
Since this knowledge is unknown initially, the PMKB must hold both the expected or 

planned value for each subsystem-milestone-aspect and how that value was 
obtained. 


The data structure for such a subsystem-milestone-aspect is shown illustratively 
in Figure 6. The slots for holding the important pieces of information are shown 
however, the methods and other intelligence features are only implied by this 
Figure. The prototype EPMS implemented and tested most of these features with 
the exception of the projection capability. 


3.3) Partitioned Analog Knowledge Base (AKB) 

In order to facilitate search and to improve execution time the analoqs are 
stored in a structure (Figure 7A) that defines two important characteristics: (1) the 
typo ogy classification scheme, and (2) the progressive deepening levels The 
typology classification scheme captures the sorting process that PMs use to classify 
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projects. For example, the statement "this project is a Class X, Type Y" is often 
encountered in analogical reasoning (see Silverman 1983, 1984, 1985, 1986). Hence 
the knowledge base offers a "typology plane" or surface. This classification scheme 
can be pursued at multiple levels of problem solving depending how deeply 
involved the problem solver is. Thus a progressive deepening scheme is also offered 
wherein the EPMS user could enter at the level at which he wishes to work (only 
three example levels are shown in Figure 7 A, more are possible). 

Within each partition of Figure 7 A, are the analogs that correspond to that class- 
type of problem. Each analog is itself multi-dimensional as portrayed illustratively in 
Figure 7 B. The three dimensions shown are of variable length and capture the fact 
that most projects involve multiple subsystems each marching along a set of 
prescribed milestones. Attached to each milestone are each of the aspects listed 
vertically in Figure 7 B. The actual knowledge about each subsytem-milestone- 
aspect is stored in any of a number of possible representations (e.g., map, icon, 
graph, table, list, rule, etc.). Also stored with each subsytem-milestone-aspect are 
any relevant advice lessons learned, etc. for selected class-types of problems 
encountered.The prototype EPMS includes a 270 node object lattice of attributes 
associated with Figure 7 A and three analogs corresponding to Figure 7 B. The three 
analogs are the LANDSAT, SPACE TELESCOPE, and NIMBUS-G projects. In this lattice, 
each node represents a specific activity, system element, or organizational element 
of a given project and is organized in hierarchies and grouped into specialized 
project domain areas, so that the further the lattice is traversed, the more detailed 
the information about a specific project domain becomes. In this way, the lattice 
can be used for two purposes within the project management domain: (1) as a 
guide for selecting attributes to characterize new analogs being entered into the 
AKB, and (2) as a means of entering a description of the target problem/project to 
be planned/analyzed. 

3.4) ARIEL Subsystem 

The ARIEL subsystem physically implements the 5-part analogical reasoning 
process described in Silverman (1985) & Silverman & Moustakis (1986) within five 
specialists and a blackboard (see Figure 8). Each specialist consists of a short-term 
memory, local knowledge bases, and an interface to the main blackboard, which is 
the shared memory used by the other specialists. The local knowledge bases store 
the knowledge regarding the specialist's particular area of expertise, as well as 
knowledge related to planning and control. The local knowledge bases are used to 
formulate an approach by the specialist on the main blackboard. The local control 
knowledge base controls the flow of information between the specialist and the 
main blackboard, making sure that only relevant data/information is exchanged. 

3.4.1) The CHAIRMAN 

The main blackboard is interfaced with a CHAIRMAN that controls the 
information flow among the five specialists. The CHAIRMAN has the same basic 
structure as a specialist. The primary purpose of the CHAIRMAN is to monitor the 
operation of the five specialists and to maintain an orderly flow of information to 
and from the blackboard. The CHAIRMAN also handles all inputs and requests from 
the user. The situation can best be compared to an individual (the user) presenting 
a problem to another individual (the CHAIRMAN). The CHAIRMAN then convenes a 
meeting of five specialists each of which is an expert in a particular aspect of the 
application of analogical reasoning to problem solving in general. These experts 
are sitting at a conference table (the Blackboard) and the input problem/request 
from the user is placed on the table by the CHAIRMAN. The CHAIRMAN directs each 
specialist to look at the information on the table. Each specialist is asked by the 
CHAIRMAN to prepare and submit a plan to solve or to help solve the problem, 
based on its own local knowledge about problem solving. The CHAIRMAN then 
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evaluates each plan and decides which plans are to be activated. The activation of 
one or more plans will result in new information being presented to the blackboard, 
which could conceivably affect the planning processes and results of the other 
specialists. The CHAIRMAN'S main purpose, therefore, is to decide which specialists 
should be permitted to proceed and in what sequence in order to maintain an 
optimal and orderly progression toward the goal (target solution). For the purposes 
of EPMS, a plan submitted by a specialist to the CHAIRMAN is called a SAR (Specialist 
Activation Request), and the order issued by the CHAIRMAN to a specialist to 
proceed with that plan is called an Execution Order for Specialist (EOS). The SAR is 
similar (analogous) in nature to the KSAR employed in HEARSAY-II. (Erman, et.al 
1980). 

3.4.2) The Five Specialists 

Each specialist is a self-contained expert system that opportunistically examines 
the contents of the blackboard and proposes an analogical reasoning related 
process or function (e.g., diagnose, classify, evaluate, scan, assimilate, etc.) to the 
CHAIRMAN as a means of contributing to the progression of the problem toward a 
final solution. The five specialists support the problem identification (PI), 
Knowledge Acquisition (KA), Analog Transfer (AT), Knowledge Transformation 
(KT), and Introduction into Use (I) steps outlined in earlier articles, and are called the 
CRITIC, LIBRARIAN, IDEAMAN, CRAFTSMAN and WRITER, respectively. 

The main function of the CRITIC is to aid in the process of problem identification, 
problem formulation and requirements definition. To this end the CRITIC monitors 
the contents of the target problem definition space and determines what methods 
are to be employed in order to expand or refine the target problem definition. 
These methods usually entail the selection of an appropriate problem definition aid 
being presented to the user (via the Writer). The CRITIC is also charged with the 
overall responsibility of monitoring the target solution generation process as a 
whole. These tasks range from seeking additional information from the user or 
LIBRARIAN to invoking a "stopping rule" when either an optimal solution has been 
achieved or when successive iterations would produce little or no change in the 
entropy of the target solution. The purpose of the LIBRARIAN is to ensure that all 
possible building blocks within a certain threshold that could be used in 
constructing a target solution to the problem are being considered. In order to 
accomplish this, the LIBRARIAN conducts a search of the AKB by taking each 
attribute contained in the target problem definition space, searching for each new 
occurrence of that attribute, and returning to the blackboard all previously 
unconsidered bases exhibiting that particular attribute. Another major task of the 
LIBRARIAN is to ensure that the AKB is properly updated with new information 
generated either by the user or by the ARIEL system itself. Currently the LIBRARIAN 
is configured only to assimilate final results as a new base (analog) to be considered 
for subsequent problem solving sessions. In later versions of ARIEL it is planned to 
also incorporate intermediate results, including erroneous paths, etc., in order to 
increase the overall intelligence of the system and to promote maximum reuse of 
lessons learned during the problem solving session. 

The primary responsibility of the IDEA MAN is to evaluate each candidate analog 
based on the value of the similarity metric [Silverman 1986] for that particular 
analog and the corresponding attributes contained in the target problem 
definition. Weighing factors to be used in calculating the similarity rating are 
provided by the user at the request of the IDEA MAN via the WRITER. The candidate 
analogs are ranked starting with the analog having the highest similarity rating, 
along with the value of the rating. This output represents a prioritized and 
valuated space of potential solutions for use by the CRAFTSMAN in generating a 
composite target solution. 
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The CRAFTSMAN has as its goal a means ends analysis that leads to the 
construction of an optimal solution to the target problem using to the qreatest 
extent possible the existing analogs contained in the knowledge base and provided 
bv the user. At this point in the process, all relevant analogs that have been 

nfrSiwrt en e * a ' ua V ed u nd ranl f ecl - ln constructing the target solution, the 
n TSMA u S J artS Wl i! h the ^'9^^ ranked analog and checks for a similarly value 
of 1.0, ' n ^nich case that analog becomes the final solution and the stopping rule is 
invoked by the CRITIC. If the similarity ratinq is less than 1 .0, the CRAFTSMAN takes 
the analog with the next highest rating, and constructs a temporary target solution 
by combining it with the highest-ranged analog. At this point a new similarity 
rating is calculated and compared with the rating of the highest-ranked analoq If 

?. ' S j® cond ” hi 9 he ^ t candidate is dropped from consideration 

and the third highest candidate is considered in a similar fashion. If the new ratinq 

IS the tem P° rar y target solution now becomes the new basis for comparison 

and the process continues. H 

The WRITER is the only element through which ARIEL can send messaqes to the 
user. For this reason, most of the functions that are assigned to the WRITER involve 
the generation of prompts aimed at soliciting user input. The core structures of 
lufo « res,< ^ e ' n the ARIEL knowledge base and are accessible via the 
LIBRARIAN Once accessed, the WRITER fills in any additional information germane 
to the problem being worked, and outputs the resulting prompt directly to the user 
vi3 the screen. These prompts contain hooks to specific data structures (lists) on the 
blackboard and these lists are automatically updated as the user responds to the 
prompt. The other main function of the WRITER is to document ARIEL results and 

fu° aSd 1 solvin 9 activity in a manner acceptable for inclusion by the LIBRARIAN into 
tne akb. 

3-4.3) The Blackboard Problem-Solving Framework 

The blackboard configuration provides an opportunistic search framework that 
is used to support an orderly progression from initial problem definition to final 
S ° *i tl0n £° rrr, ulation (see Figure 9). ARIEL differs from most conventional 
blackboards in that the user, at his option, can override the actions of any specialist 
at a A^,i S Jf 9 . e m the problem solving process. This was considered necessary in order 
for ARIEL to truly function as an "extension" of the human analogical reasoning 
process. The arrows in Figure 9, which indicate lateral, forward or backward level 
transitions show the user as being totally unconstrained. The CRITIC has the next 
greatest influence on effecting changes in the direction of the problem -to-solution 
progression. Note also that this process is iterative, and can be influenced by the 
activity of the other specialists. y 

3.5) User Session 

The current prototype has the capability to run three exemplary user sessions 
that were developed with the intention of soliciting feedback from potential users 
and providing additional insights into the design of the overall system. In 
particular, the sessions were intended to address some of the EPMS Research 
Questions especially with regard to the use of analogy, and the determination of 
what analytical techniques should be directly incorporated into EPMS. The three 
prototype sessions and the objectives of each are delineated as follows: 

1) Session 1, to demonstrate the project requirements definition function and 
the use of the analogical reasoning extension. 

2) Session 2, to demonstrate the budget planning function as supported bv 

analytical techniques. y 
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3) Session 3, to demonstrate the project monitoring function as supported by 
analytical techniques, including the cross referencing capability of the EPMS 
knowledge base. 

A step by step summary of each of the sessions is provided in Table 2. 

3.5.1) EPMS Control Panel 

The purpose of the EPMS Control Panel is to provide a total multi-screen 
environment in a combination desk-top/pilots’ console configuration from which 
the user can access any and all functions during an EPMS session. The control panel 
must also facilitate smooth-flowing user-machine dialogs. Use of the mouse/cursor 
is favored over the keyboard whenever feasible. Although the control panel is 
ultimately intended to be "tamper-proof", locking out all unauthorized access to 
the EPMS executive or resident software, the prototype version allows this access 
because of the developmental nature of the system. 

The EPMS control panel isdisplayed in Figure 10. Some of the features were fully 
implemented in the prototype, others appear on the screen but are currently 
inactive (specifically the calendar, CVE screens and manual mode). 

4.0) EPMS Prototype Evaluation 

The purpose of this section is to document the results of the EPMS prototype 
evaluation sessions. Following a sequence of software IV & V (Initial Validation and 
Verification ) tests, a series of expert evaluation sessions were conducted in which 
potential users were given the opportunity to run exemplary sessions and to provide 
reactions/comments on the overall system design and user interface. The comments 
were then used to create a composite summary of desired 
enhancements/improvements to be incorporated in EPMS Version 1.0, and to 
generate a list of 1 1 research questions mandating continued further exploration. 

4.1) Expert Evaluation 

This section describes the results of six separate knowledge collection sessions in 
which domain experts were given demonstrations and/or functional descriptions of 
EPMS: i.e., its goals, its conceptual design, and the types of sessions a user would 
encounter. The domain experts, in turn, each offered several types of feedback that 
are documented here including: 

1) Evaluation of EPMS in terms of its goals, design and sessions. This feedback 
included suggestions for altering and improving EPMS. 

2) " Deep Knowledge" was offered for EPM's knowledge banks. That is 
numerous ways for EPMS to utilize and/or "plug in" to existing handbooks, data 
bases, and other procedural aids was offered. 

3) Heuristic Knowledge and rules of thumb used in project management were 
elaborated that could and should be incorporated into the EPMS knowledge 
base. 

These three types of feedback-evaluation, deep knowledge, and heuristic 
knowledge are popularly thought to be collected in distinct sessions: a knowledge 
elicitation session with domain experts and an expert system evaluation session by 
potential users. While textbook descriptions of knowledge engineering invariably 
separate evaluation from elicitation, the fact is that both forms often are 
intermingled in any one interview session, particularly so during the conceptual 


design period, as was the case here. A summary of the types of sessions and the 
experts participating in each is provided in Table 3. 

4.1.1) Overview of 17 Experts' Comments 

„_T- ab '? j* P. rovides a comprehensive summary of the major comments/suggestions 
H^n ^ d + h rm9 + e + 6 sess 'P ns ' and indicates whether the suggestion influenced the 
design of the prototype, the user sessions (interface), the planned version 1 system 
-term enhancements. Not surprisingly, the three unanimous reactions 
V the need for an environment to support and extend the human expert 
analogical reasoning process, 2) the need to structure domain-related knowledge in 
a l-ciimensional format that supports traversal across various functional 
characteristics and down various levels of granularity (progressive deepenino)- 3) 

man^n^hf 0 Ca + ptur + " la s sons leamed " and incorporate this knowledge^ a 
9 ! b J e aut °^ atGd Co ;p. orate Memory" structure, that could be called on to 
a ? V Ji C ?h W * ber !u S u X ° f similar conditions are detected in real-time. It should 

fiSnn i°? te a d tb !i wlth !g h ! exce P tlon of the executive assistant concept arrived at in 
session #2, and the need for a separate AKB and PMKB from session #3 no other 

+ mque to only one session. In fact, each session generated an 
average of about seven comments that were either actually incorporated in the 
prototype or were entered as planned enhancements for either Version I or other 
long-term developments 

4.2) Insights for an EPMS Generator 

At the beginning of the EPMS prototype effort the investigators formulated 1 1 

questions to be researched as stated in technical objectives of this report The 

purpose of this section is to delineate the 1 1 research questions (see Table 5) and to 

discuss the answers arrived at during the course of the study. The first three 

tha+t^ rS ind J Ca * t l h ? t there is a strong-felt need for analogical support; maintaining 

dPdiSt P pH°^c S ct OU t d H 0t iS q k Uire mucht '[ rie ° r effort of the project team members, a 
dedicated assistant should be responsible for interfacing to EPMS (it may not be his 

only responsibility). The next two answers repeat the fact that there are an 

tU'ihlf f ab 6 n i Jmbe , r of f M subsystems possible, most of which should be relegated 
to the longer term development period orto user development activity. 

In terms of the Physical Model, answers to questions 6 through 8 were explored 
m describing the design of the Control Panel and sample uslr sessions and are 

?h^dl a Jr^ d m T u ble xu' primari| y i " terms of utilities and packages needed to effect 
ctjfrtinn T * tS ‘ These are not final answers but rather may be viewed as good 
f ° r , 4 fu ! U / e r ®J ,nement - The answer to question 9 extends the scope 
of the utilities needed for effective user interrelation with EPMS. Finally the answer 

generality 0 10 P ° mtS tQward utllity P acka 9 es that help EPMS achieve flexibility and 

HpuIkfr> V c3\/!c St q T tl0n u deals with what machine < environment and language to 
# e ^ e !u P EPMS Ir l- This IS t h e same question numerous software vendors have vet to 
I'r« 0 the ,? ptimal answer to. The only solution seems to be to develop the product 

Is t"me anSfLndfper m°t" 9radu3 " y POr * * ,0 ° ,her machines 

5.0) Next Steps 

Given the work done to date, the lessons learned, (partial) answers to the 
research questions, and the user interests and attitudes, a number of next steps are 
immediately obvious. These include: af e 

^ Exceed With ARIEL Subsystem - The ARIEL SubsystPm <hrmH be flushed out 
as soon as possible and as originally designed and conceived. No user reactions 
indicated any concerns about ARIEL'S design or heuristics. On the contrary, users 
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logical *ooe 1 

(1) \i . is analogy t r w ! jf important to project managers joa t? j<j 
w*at is Us wst important purpose’ Is there a hierarch,, 
of purposes and uses? „ 


(2) y. Given the rapid pace of most projects and the already heavy 
staff workloads, no. snoulo analogy information be collected 
for use in £PMS? 


. ne progressive aeepen-.-g concept was 

uniformly accented as me correct -ay 
to organize each analog arc nence. this 
hierarchy snouid include executive, 
management, ano technical levels. 

4. Some 'orm of Customizing utility for 
defining : ne structure,'? ormat of the 
user s analogs should p* prepared. 


(3) 0. Given that a project is « team of people should there be a 
Single principal user o' EPHS or should E P*S simultaneously 
support a number of memoe"s of the team’ 


A, EPNS should be oriented arp u na two 
tiers of management (manager and iub- 
managtrs) with a dedicated executive 
assistant as the prime interface 
agent . 


(*) Q. How exactly will these different users utlize EP*$’ The 
analogical information’ wnat types of sessions -ill mey 
•ant to nave? 


(6) 0. Analogical reasonn 
wnich closely relai 
Into £P*S VS. whici 


»g is ontj one tecnn'que 
:eo techniques snould be 
s should be relegated to 


of project personnel. A. 
directly incorporated 
user integration efforts? 


Physical Hoool 


The details of EPMS use will depend on 
the character i st 1 c s of the particular 
subsystem being installed, however 
it will most lively include both 
planning and monitoring modes. 

There are an inumerjoie set of related 
techniques utilized in each stage of a 
project’s life. It appears that ill 
of these should be relegated to user 
development and integration, n least 
m the short term. 


(6) g. A great many heuristics (i.e., computer i zed functions) appear 
necessary so that non-progr inr*<"s can effectively retrieve 
ana manipulate anaogy i"d project data, mhicn of these are 
most Important’ In wnat orger should they be developed’ 


(7) Q. Nan machine •nte'-'aces can ^»e a large difference m tne 

success of a software pacia^e. ho* should the use' Mer'ece 
be designed’ wnat »<m:s :♦ “me tows s" i o«i'! me jSt r see cn 
tne screen and “ow vio^id ^se r access be effected’ 


(d) J. Along tne same <’n es , s"Ou J :n e jSf r !ftt erfjce he 
designed from * mowi*^* r epresentat i on and dismay 
perspecc 1 ve’ 


A. 


in each of tne demonstrations given, 
mu 1 1 * -wi ndo-ed . visual feeooac« seemed 
popular and easy to comprehend, how- 
ever, the exact details of »nat snoulo 
be contained in each window, «en u bloc* 
require much greater user testing. 


a. * "umber of knowledge represent at 1 on i;r« s 
Should be ut ' 1 1 zed ’ncluO'hg ru’es. 
latfces, frames, tables, lists, an.: 
’mages. TaOles. lists ana -mages 
appeared most popular for output d t,s 
Pou-up menus were the desired mode i> 
vser input, except a “latt'te cm;.**-' #<> 
preferred f 0P ^re complex -somam*. 


*. 'he user interface ns/Sl Support 

generation of relat’vfiy numerous s 
simple typology c a tegor i za t ’ a* s . 


1*1 


( 1 U) 


Nhat IS needed to assist "onprogrammer * in CjS 1 0*1 Z ’ £ vx<, 
to their site? wnat can oe expected to oe generic vs. wnat 
is sue specific m xn^iea^e bases anq a-uEi subsystems’ 

Nhat customi zing /extenq 1 ng utilities are needed ana m «nat order 
should they be developed’ 


*. The experts' feedback seems to >-.j- , , 
that every aspect of EP*S snouia ■» 
designed with a customizing a>d -- 
fr/ym me screen elements, to me *•>* 
structure, to the p* SuOSyStem -t-r'itn. 


— 5 ^ j* eiywnofa to oeveioy the project 

management subsystem’ •" » ^seM generally nave them own stand 
alone aids that mey want to connect £P«S to or «».i mey „ <A t 
EP« to provide mis capability? wnat is me essential *,rn f i 
Of this Subsystem that many types Of potential Ulerj will find 
most effective’ wnat customizing utilities wilt be needed to 


In the Short tern it may be nv’l fru>: # ji 
to concentrate all effort ana ( -ir, 

AfllEl. me utilities, ana the i«c#r'a.e” 
subsystems. 


Actual Computer Imp tementat 1 on 


(II) y. Ideally, me £ p ms generator snould be readily available to users 
wfio own a variety of vendor workstations and colters , ui*en 
that tne prototype >s being developed on a teroi Lisp macnme 
is It reasonable to n*«f maximum uSt of COMMON LISP and of 
18* PC interfacing software or is some omer language and 
environment ',e.g.. C 0 « me SUh’ more nearly optimal? 



A, There is no easy answer to tvs question 
as numerous software vendors nave disco- 
vered. For tne P* world one would expect 
that purchasing 1 stand-alone workstation 
to r u n £p*s snoulo not oe a large drain on 
meir ouagets, especially with the prtr e 
of workstation* falling so rapidly, now- 
ever, if EP*S is to be wdeiy used, the 
requirement to buy a l!SP machine seems 
prohibitive, particularly m view of the 
lesson learned that many users Will wish 
to integrate £P*S with some Custom 
designed stand-alone Ph suosystem. Tn* 
danger of locking tn to th* «achm, 

world is that the Overwhelming majority 
of PN offices have [BN PC’s or any 0 t 4 
host of other workstations or minicompu- 
ters but few of them have UiP machines 
In me s«ort run this obstacle prooaoiy 
can t be overcome. In the mid term me 
solution might be to develop “bridging* 
soft-are that creates virtual interfaces 
to a variety 0 f other machines (e.g., ism 
pc world) on which many stand alone rw 
subsystems are likely to be built, a 
1o«9er term solution is to port £P*C to 
1 **«©er of different machines that 
Support coxmon LISP and/or to port it ig 
a second language such as C which is 
fai r ly avai t ib t e. 


want lots of simple heuristics, progressive deepening, typology and level 
selection, etc. 

2) Flush Out Manual and Interrogator Mode Utilities ~ These utilities have been 
defined and should now be built to permit users to both inspect all aspects of 
ARIEL reasoning chains and to permit advanced users to manually reason by 
analogy on the KB elements. 

3) Develop Customizing Utilities-- Requirements for generality and flexibility can 
be satisfied with the development of numerous, relatively small utilities. Each 
utility can perform a single adaptation function (e.g., support individual analog 
feature generation tasks) that permits EPMS to be molded to the user's specific 
PM subsystem needs. 

4) Select Field Test Site(s) -- Until users begin to actually try and apply EPMS to 
their site and to use it on a regular basis, there will be no way to accurately 
evaluate its man machine interface. For that purpose, one or more test sites 
should be selected as soon as possible and EPMS should be installed and adapted 
to their problem(s). It is most desirable to select a test site that either already has 
an existing stand alone PM subsystem or that does not want a very strong PM 
subsystem capability. These sites would provide useful MMI insights with a 
minimum of tangential PM subsystem development activity. A longer term goal 
will be to select sites with needs for greater PM subsystem help so as to focus in 
more clearly on what the customizing utilities and kernel elements should entail. 
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